The invention relates to the field of message transfer. The invention is described in the context of the messaging system, MQSeries Integrator provided by International Business Machines Corporation (MQSeries is a trade mark of International Business Machines Corporation). However, the invention can be used in any situation or environment in which messages are transferred.
MQSeries applications can pass messages between queues in applications, which can be located anywhere in a customer's enterprise. Applications can also exchange messages with applications from a customer's clients in locations remote from the customer. As customers look to do more complex tasks and to put their data to more productive use, there is a need to process and transform the message data before passing the message on to the next application.
A message broker provides a hub for processing, transformation and distribution of messages. Message brokers act as a way station for messages passing between applications Once messages have reached the message broker they can proceed, depending on the configuration of the message broker and on the contents of the message.
The message broker has a message flow framework including publish and subscribe management and message storage resources. A control center of the message broker controls the message flows. Within the message broker, the individual functions are assigned to a collection of interconnected constructs called nodes, where the processing and transformation activities can take place as required.
The MQSeries Integrator product supports the application communication model known as publish/subscribe. In this model, applications known as publishers send messages and others, known as subscribers, receive messages. Applications can also be both publishers and subscribers. The publishers are not interested in where their publications are going, and the subscribers are not concerned where the messages they receive have come from. The broker assures the integrity of the message source, and manages the distribution of the message according to the valid subscriptions registered in the broker.
The interactions between a broker and its publisher and subscriber applications are equally valid in a broker network in which publish/subscribe applications are interacting with any one of a number of connected brokers. Subscriptions and published messages are propagated through the broker domain. Brokers can propagate subscription registrations through the network of connected brokers, and publications can be forwarded to all brokers that have matching subscriptions. When the term “broker” is used it generally includes a single broker or multiple brokers working together as a network to act as a single broker.
A message flow running in the message broker retrieves a publication from its input mechanism, performs any processing that is defined for publications received in that message flow and passes the message to a publication node for distribution to one or more subscribers. The input and output mechanisms of the broker may be in the form of asynchronous connections such as input and output queues, synchronous connections such as input and output nodes with direct client connection via TCP/IP connections, a combination of such input and output mechanisms, or other connections via mobile telecommunication, email etc. All these forms of input and output to and from the broker are referred to generally as input and output mechanisms.
The publication node only knows about, and can therefore only provide messages to, an application that has registered as a subscriber. When the application registers, it must specify a means by which it wants to receive messages (which may be a queue or other input mechanism) and a definition that restricts the messages it wants to receive. The definition can be based on a combination of the topic of the message or specific content within the message or both.
In some situations, it is desirable to make the messages between publishers, subscribers, and a message broker as small as possible. This is often because of low interconnection bandwidth, or high cost per byte for data transmission between remote client computer systems. Value can be gained from compressing information sent in a message. For example, in remote telemetry in which distant events are measured and the data transmitted in the form of a message, every byte transmitted has a cost.
In such cases, it is common to compress all or some of the message before transmission from publisher to broker and to decompress it in the broker to allow it to be inspected to find out essential information, like the topic name, and other publication parameters such as quality of service, etc. The most likely candidates for compression are the topic name and the message “payload”, which is the body of the message which is being delivered. The topic name is often quite verbose, in a form like a uniform resource locator, in a potentially detailed topic space, for example, “weather/location—1/humidity/indoor/high”.
After matching against the subscription lists of subscriber clients, the message is typically recompressed before being sent out to those subscribers who have expressed an interest in the particular topic of the message.
In other situations, messages are encrypted before sending between publishers, subscribers and a message broker in order to ensure that the message is not intercepted and read by a party other than the intended recipient of the message. The encryption may result in the message being longer than the unencrypted form.
This compression, decompression and recompression or encryption, decryption and re-encryption is time consuming, processor intensive, and diminishes the performance of the broker in terms of total message throughput rate. This invention proposes a solution to this problem. An aim of the invention is to provide efficient processing of compressed or encrypted messages when they arrive in a message broker.